Knowledge Acquisition for Performance Systems; or: When can "Tests" Replace "Tasks"?

Currently, "task analysis" is the dominant paradigm in the knowledge acquisition community. We argue that for performance systems (i.e. systems that do not have to offer a knowledge-level description of their performance at runtime) a simpler "test analysis" approach may suffice. We offer examples were a seemingly-naive testing regime gives rise to competent performance systems. Further, by certain measures, these systems developed via test analysis out-performed systems developed for similar domains using other techniques. Test analysis did not augment some other methodological approach: it removed the need for any other methodology. We speculate that for performance systems, task analysis could be deferred till after the development of a tested performance system. That is, for performance systems, testing replaced task but task analysis could augment test analysis once a system was in production.